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AIRBORNE  PERFORMANCE  MEASUREMENT  METHODOLOGY 
APPLICATION  AND  VALIDATION:  FA  POP-UP  TRAINING  EVALUATION 


I.  INTRODUCTION 

Recently  the  availability  of  energy  and  aircraft  resources  for  Air  Force  flying  training  has 
decreased  dramatically.  As  these  resources  continue  to  diminish,  the  demand  increases  for  more 
effective  use  of  aircraft  sorties  and  training  aids.  In  response  to  this  demand,  the  Air  Force  has 
initiated  a flying  skills  maintenance  and  reacquisition  training  research  program  (Project  SMART). 
The  objective  of  Project  SMART  is  to  identify  and  define  the  critical  combat  skills  of  mission 
ready  aircrews  and  to  develop  procedures  for  measuring  these  skills.  The  resulting  skill 
measurement  procedures  will  enable  Air  Force  training  managers  to  increase  the  efficiency  of  their 
programs,  to  evaluate  aircrew  maintenance  of  critical  combat  skills,  and  to  provide  mission  ready 
aircrews  at  minimum  cost.  The  skill  measurement  procedures  will  be  used  to  identify  areas  of  skill 
deficiency  and  to  assess  the  effects  of  modifications  in  flying  training  programs.  The  result  of  this 
research  effort  will  be  recommendations  for  improving  initial  and  continuation  training  programs 
based  on  behavioral  data  and  documented  flying  requirements. 

In  its  current  phase,  Project  SMART  has  focused  on  the  identification  and  definition  of 
critical  skills  required  for  selected  air-to-ground  weapon  delivery  maneuvers  and  on  the  development 
of  a methodology  to  measure  these  skills.  The  pop-up  weapon  delivery  maneuver  (pop-up)  was 
identified  as  one  of  the  tasks  critical  to  mission  readiness  and  was  chosen  for  initial  study.  The 
pop-up  requires  a low  altitude  approach  to  a pull-up  point  selected  on  the  basis  of  its  position 
relative  to  a target  (see  Figure  1).  At  the  pull-up  point,  a specific  climb  angle  is  established  to 
achieve  enough  altitude  to  acquire  the  target  and  set  up  the  bomb  run.  At  the  appropriate 
altitude,  the  pilot  rolls  in  on  the  target,  apexes,  rolls  out,  establishes  the  correct  dive  angle,  and 
flies  the  aircraft  to  the  release  point  required  for  ordnance  delivery.  Upon  ordnance  delivery,  the 
pilot  takes  action  to  evade  ground  threats  and  return  to  low  altitude.  (For  a more  complete 
discussion  on  the  pop-up  delivery  and  its  use  in  the  tactical  environment,  see  Dyches,  1978.)  The 
pop-up  is  designed  to  minimize  the  amount  of  time  spent  outside  the  low  altitude  environment. 
By  minimizing  the  time  he  is  exposed  to  ground  threats  during  weapons  delivery,  the  pilot 
increases  his  survivability.  Thus,  the  pop-up  is  a time-compressed  tactical  maneuver  which  requires 
a variety  of  heads-up  advanced  flying  skills  for  its  execution. 


Figure  1.  Pop-up  weapon  delivery  profile. 
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At  the  start  of  the  research,  extensive  interviews  were  conducted  with  mission  ready  pilots 
to  identify  and  define  the  skills  critical  to  pop-up  performance.  The  interview  data  suggested  that 
pilot  planning,  cross-check,  discrimination,  anticipation,  decision  making,  and  aircraft  control  skills 
were  critical  to  successful  performance  of  the  delivery.  The  more  proficient  pilot  focuses  his 
attention  ahead  of  his  aircraft,  increasing  both  his  situation  awareness  and  his  ability  to  make 
rapid  error  analysis.  In  developing  a methodology  for  performance  measurement,  heavy  emphasis 
was  placed  on  procedures  to  assess  cognitive  skills  through  the  use  of  behavioral  data. 

Published  procedures  for  the  pop-up  provide  precise  quantitative  specifications  of  aircraft 
control  parameters.  Since  means  for  determining  the  values  of  critical  aircraft  parameters  during 
the  pop-up  were  not  available  for  use  in  evaluating  pilot  performance,  a performance  measurement 
methodology  was  developed  which  took  advantage  of  the  pilots’  ability  to  assess  performance  on 
the  critical  stages  of  the  maneuver.  A form  listing  these  critical  stages  in  sequence  was 
constructed,  on  which  pilots  could  rate  performance  for  data  collection  purposes.  Skill  levels  of 
the  cognitive  activities  associated  with  each  stage  were  inferred  from  the  data.  This  procedure 
resulted  in  the  identification  of  basic  areas  of  performance/non-performance. 

The  present  study  addresses  the  validity  and  applicability  of  the  performance  measurement 
methodology.  Pilots  in  an  F4  combat  crew  training  program  served  as  subjects.  This  procedure 
increased  the  availability  of  subjects  and  permitted  the  use  of  instructor  pilots  (IPs)  to  assess  pilot 
performance. 

Objectives 

The  present  study  was  designed  to  validate  the  pop-up  performance  rating  methodology.  In 
addition,  the  study  generated  information  identifying  specific  areas  of  pilot  performance/ 
non-performance. 


II.  METHOD 

Subjects 

Twenty -one  F4  B-course  pilots  assigned  to  the  311th  Tactical  Fighter  Training  Squadron, 
Luke  AFB,  served  as  subjects.  All  subjects  were  undergraduate  pilot  training  graduates  and  had 
completed  fighter  lead-in  training  but  had  not  been  qualified  in  any  other  operational  aircraft.  At 
the  time  the  study  was  conducted,  subjects  had  just  completed  the  ground  attack  (GA)  and 
ground  attack  radar  (GAR)  phases  of  the  training  syllabus.  The  GA  and  GAR  phases  concentrated 
on  training  and  qualification  in  basic  bombing  maneuvers  and  nuclear  deliveries,  respectively.  The 
ground  attack  tactical  (GAT)  phase  followed  the  GAR  phase  and  introduced  the  pilot  to  advanced 
tactical  weapon  deliveries  including  curvilinear  and  pop-up  approaches.  Thus,  the  pop-up 
performance  data  obtained  in  this  study  were  collected  on  pilots  who  had  not  previously 
performed  the  pop-up  delivery.  The  GAT-phase  concluded  B-course  training,  at  which  point  pilots 
were  considered  “mission  ready”  and  assigned  to  operational  units. 

GAT  Phase  Description 

The  GAT  phase  consisted  of  six  sorties  (GAT-1  through  GAT-6)  which  typically  were  flown 
in  four-ship  formation.  In  the  event  of  a ground  abort,  the  remaining  members  of  the  flight 
continued  the  mission  in  three-ship  formation.  Sorties  GAT-1  through  GAT-3  were  flown  on 
controlled  ranges.  Mission  planning  for  these  sorties  was  conducted  by  the  flight  IPs.  Sortie  GAT-1 
served  as  an  introduction  to  curvilinear  and  pop-up  approaches  and  high  angle  strage,  while  sorties 
GAT-2  and  GAT-3  focused  on  the  pop-up  and  provided  the  pilot  opportunity  to  become  crew-solo 
qualified.  Sorties  GAT4  through  GAT-6  were  flown  on  tactical  ranges.  Sortie  GAT4  introduced 
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the  pilot  to  planning  and  flying  pop-ups  on  tactical  ranges.  On  sortie  GAT-5,  the  pilots  were 
required  to  plan  missions  simulating  a high-threat  environment  and  to  brief  the  procedures  of  the 
flight  to  mission  aircrew  members.  The  GAT  phase  was  concluded  with  sortie  GAT -6  in  which  a 
mass  attack  of  12  to  16  aircraft  was  conducted  in  coordination  with  other  F-4,  F-104,  and  F-15 
squadrons,  simulating  a high-threat  combat  environment. 

The  number  of  pop-ups  performed  on  each  sortie  varied,  depending  on  whether  the  sortie 
was  flown  on  a controlled  or  a tactical  range.  Generally,  each  pilot  performed  four  to  eight 
pop-ups  on  controlled  range  sorties  and  two  to  four  deliveries  on  tactical  range  sorties. 

Subjective  Rating  Form 

Pilot  performance  ratings  were  collected  using  the  F-4  Pop-Up  Evaluation  form  (see  Appendix 
A).  The  form  segmented  the  pop-up  into  nine  (controlled  range)  or  ten  (tactical  range)  critical 
stages.  Performance  on  each  stage  was  rated  using  a three-point  scale  (Satisfactory-Marginal- 
Unsatisfactory).  Written  explanatory  comments  were  required  for  all  marginal  and  unsatisfactory 
ratings. 

Bomb  scores  and  estimated  bomb  release  parameters  were  recorded  for  each  delivery.  The 
best  available  objective  measure  of  pop-up  performance  was  the  circular  bomb  error  or  bomb 
score.  Bomb  score  data  were  collected  for  use  in  validating  the  subjective  ratings  obtained  on  the 
form. 

Procedure 

Procedures  for  data  collection  were  described  to  all  IPs  participating  in  the  study  at  the 
GAT-phase  squadron  briefing.  Pilot  performance  was  assessed  and  recorded  by  IPs  at  the  debriefing 
of  each  mission.  Stage  performance  requirements  and  rating  scale  descriptions  were  provided  in  the 
instructions  on  the  reverse  of  each  form  (see  Appendix  A).  A researcher  was  available  at  all  times 
to  hand  out  and  collect  the  data  forms  and  to  answer  questions.  Due  to  the  non-interference 
nature  of  this  study,  manipulation  of  normal  squadron  operations  was  impossible.  Therefore,  data 
collection  procedures  were  integrated  with  normal  flight  operations  throughout  the  GAT  phase. 
Pop-up  performance  ratings  were  collected  on  all  deliveries  performed  by  pilots  throughout  the 
phase. 


m.  RESULTS  AND  DISCUSSION 

Validation  of  the  Performance  Measurement  Methodology 

In  order  to  make  inferences  about  flying  skill  from  the  subjective  ratings,  it  was  necessary  to 
demonstrate  that  the  ratings  are  related  to  the  best  available  objective  measure  of  the  quality  of  a 
pop-up.  In  this  instance,  the  validity  of  the  rating  methodology  was  determined  by  using  a 
composite  of  the  individual  stage  performance  ratings  to  predict  bomb  score. 

In  constructing  the  composite  performance  rating  score,  it  was  assumed  that  performance  on 
each  stage  of  the  pop-up  is  relatively  independent  of  performance  on  the  preceding  stage  when 
that  stage  is  performed  with  little  error.  However,  gross  error  during  a stage  greatly  impairs 
performance  of  subsequent  stages.  For  example,  consider  stages  4 and  5,  target  acquisition  and 
pull-down  point,  respectively.  As  long  as  the  target  is  acquired  in  a timely  manner,  pull  down 
could  be  executed  even  if  the  target  acquisition  stage  was  executed  poorly.  However,  if  the  target 
was  not  successfully  acquired,  or  was  acquired  after  a long  delay,  performance  on  the  pull-down 
stage  would  be  impaired  by  the  need  to  correct  for  errors  in  the  earlier  stage.  Thus,  for  purposes 
of  scoring,  it  was  assumed  that  a rating  of  unsatisfactory  on  any  stage  of  the  delivery  precluded 
satisfactory  execution  of  subsequent  stages,  and  scoring  of  the  delivery  was  terminated  at  that 
stage.  A rating  of  marginal  was  assumed  to  reflect  performance  which  was  adequate  to  permit 
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satisfactory  execution  of  the  next  stage  of  the  delivery  but  was  below  the  criterion  of  acceptable 
performance.  It  was  given  a value  of  zero  and  scoring  continued  through  subsequent  stages.  A 
rating  of  satisfactory  was  assumed  to  reflect  acceptable  performance  and  was  given  a value  of  one. 
The  total  composite  score  was  obtained  by  adding  the  value  of  the  rating  for  each  stage  in 
sequence  until  a stage  was  rated  unsatisfactory  or  the  delivery  was  completed.  Thus  the  range  of 
scores  was  zero  (approach  to  pull-up  point  (PUP)  rated  unsatisfactory)  to  nine  on  the  controlled 
range  or  ten  on  the  tactical  range  (all  stages  rated  satisfactory). 

Consider  for  example  the  two  pop-ups  shown  in  Figure  2.  The  scoring  of  delivery  A would 
proceed  as  follows: 


Stage 

Rating 

Score 

Total  Score 

1.  APP  to  PUP 

SAT 

1 

i 

2.  PUP 

SAT 

1 

2 

3.  CLIMB  LEG 

MAR 

0 

2 

4.  TGTACQ 

SAT 

1 

3 

5.  PULLDOWN 

UNSAT 

STOP 

3 

Delivery: 

A B 

Task  Evaluation 

1.  Approach  to  PUP  _S _JS 


2.  PUP  S _S 


3.  Climb  Leg 


4.  Target  Acquisition 

s 

H 

5.  Pull  Down  Point 

u 

S 

6.  Apex 

s 

s 

7.  Track  Point 

s 

M 

8.  Bomb  Run  S M 

9.  Recovery  _S _S 

10.  Rtn  to  Low  Alt  _S _S 

Figure  2.  Composite  scoring  method  — example. 

In  scoring  delivery  B,  which  had  no  unsatisfactory  stages,  one  point  would  be  assigned  for 
each  rating  of  SAT  (Satisfactory),  thus  giving  a total  score  of  7.  Therefore,  even  though  delivery 


A had  a greater  number  of  satisfactory  ratings  than  delivery  B,  it  received  a lower  score  due  to 
the  unsatisfactory  rating  early  in  the  delivery.  Bomb  scores  were  grouped  into  seven  categories:  (a) 

less  than  50  feet,  (b)  50  to  99  feet,  (c)  100  to  149  feet,  (d)  150  to  199  feet,  (e)  200  to  299 

feet,  (f)  unscorable  (300+  feet),  and  (g)  aborted  pass  (no  bomb  release  for  reasons  of  safety). 

The  relation  between  composite  performance  rating  and  bomb  score  is  shown  in  Figure  3.  It 
can  be  seen  that  more  accurate  bombs  are  given  higher  subjective  composite  task  ratings.  The 

mean  rating  for  bombs  in  the  0 to  49-foot  category  was  8.2,  while  the  mean  rating  for  aborted 

passes  was  only  3.  The  magnitude  of  the  relationship  was  assessed  with  a chi-square  contingency 
coefficient  (Siegel,  1956).  Bomb  score  error  and  composite  rating  score  were  found  to  be  strongly  related 
(contingency  coefficient  = .664,  P<  .001).  This  result  shows  that  subjective  pilot  ratings  of 
individual  stages  of  the  pop-up  yield  a reliable  indicator  of  the  quality  of  performance  on  that 
delivery.  An  additional  characteristic  of  the  scoring  scheme  is  that  it  assumes  errors  made  early  in 
the  delivery  are  more  critical  than  are  errors  made  late  in  the  delivery.  The  composite  scoring 
system  weights  early  errors  more  heavily  by  omitting  more  stages  from  scoring  following  an  earlier 
unsatisfactory  rating  than  following  a later  such  rating.  The  strong  relation  between  composite 
score  and  bomb  score  suggests  that  this  shift  in  emphasis  over  the  course  of  the  delivery  is  a 
reflection  of  the  actual  criticality  of  errors  within  each  stage.  More  will  be  said  about  this  point 
in  the  discussion  of  the  data  on  the  individual  stages  of  the  delivery. 


AVERAGE  COMPOSITE  PERFORMANCE  RATING  (ACPRj 

X 

BOMB  SCORE  CATEGORIES  (BSC) 


CONTINGENCY  COEFFICIENT  = .664 

P < .001 


Figure  3.  Methodology  validation  results. 

Stage-by-Stage  Analysis  of  the  Pop-Up 

Rating  of  individual  stages  of  the  pop-up  provides  detailed  information  for  error  analysis 
which  is  not  available  from  overall  objective  measures,  such  as  bomb  score.  It  provides  information 
about  which  stages  of  the  delivery  are  more  difficult,  which  improve  most  over  training,  and  how 
this  improvement  affects  the  entire  delivery. 
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A had  a greater  number  of  satisfactory  ratings  than  delivery  B,  it  received  a lower  score  due  to 
the  unsatisfactory  rating  early  in  the  delivery.  Bomb  scores  were  grouped  into  seven  categories:  (a) 

less  than  50  feet,  (b)  50  to  99  feet,  (c)  100  to  149  feet,  (d)  150  to  199  feet,  (e)  200  to  299 

feet,  (f)  unscorable  (300+  feet),  and  (g)  aborted  pass  (no  bomb  release  for  reasons  of  safety). 

The  relation  between  composite  performance  rating  and  bomb  score  is  shown  in  Figure  3.  It 
can  be  seen  that  more  accurate  bombs  are  given  higher  subjective  composite  task  ratings.  The 

mean  rating  for  bombs  in  the  0 to  49-foot  category  was  B.2,  while  the  mean  rating  for  aborted 

passes  was  only  3.  The  magnitude  of  the  relationship  was  assessed  with  a chi-square  contingency 
coefficient  (Siegel,  1956).  Bomb  score  error  and  composite  rating  score  were  found  to  be  strongly  related 
(contingency  coefficient  = .664,  P<  .001).  This  result  shows  that  subjective  pilot  ratings  of 

individual  stages  of  the  pop-up  yield  a reliable  indicator  of  the  quality  of  performance  on  that 

delivery.  An  additional  characteristic  of  the  scoring  scheme  is  that  it  assumes  errors  made  early  in 

the  delivery  are  more  critical  than  are  errors  made  late  in  the  delivery.  The  composite  scoring 

system  weights  early  errors  more  heavily  by  omitting  more  stages  from  scoring  following  an  earlier 
unsatisfactory  rating  than  following  a later  such  rating.  The  strong  relation  between  composite 
score  and  bomb  score  suggests  that  this  shift  in  emphasis  over  the  course  of  the  delivery  is  a 
reflection  of  the  actual  criticality  of  errors  within  each  stage.  More  will  be  said  about  this  point 
in  the  discussion  of  the  data  on  the  individual  stages  of  the  delivery. 


AVERAGE  COMPOSITE  PERFORMANCE  RATING  (ACPRj 

X 

BOMB  SCORE  CATEGORIES  (BSC) 


CONTINGENCY  COEFFICIENT  = .664 

P < .001 


Figure  3.  Methodology  validation  results. 

Stage-by-Stage  Analysis  of  the  Pop-Up 

Rating  of  individual  stages  of  the  pop-up  provides  detailed  information  for  error  analysis 
which  is  not  available  from  overall  objective  measures,  such  as  bomb  score.  It  provides  information 
about  which  stages  of  the  delivery  are  more  difficult,  which  improve  most  over  training,  and  how 
this  improvement  affects  the  entire  delivery. 
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low  altitude  flight  and  BFM  receive  little  emphasis  prior  to  this  training  phase.  The  performance 
on  Stage  7,  track  point,  and  Stage  8,  bomb  run,  reflects  the  level  of  proficiency  attained  by 
pilots  during  the  GA  phase. 

In  the  more  difficult  stages,  the  same  differential  effect  of  training  is  evident.  Stages  1,  2, 
and  5 show  substantial  improvement,  although  performance  on  these  stages  does  not  reach  the 
level  attained  on  stages  3 and  4.  On  the  other  hand,  performance  on  stages  6,  7,  and  8 shows 
little  improvement  over  the  course  of  the  15  passes.  In  general  then,  it  is  the  initial  stages  of  the 
pop-up  delivery  which  improve  during  the  controlled  range  phase  of  training,  and  this  improvement 
occurs  regardless  of  the  initial  difficulty  of  the  stage.  The  effect  of  the  improvement  in 
performance  of  the  initial  stages  is  reflected  both  in  bombing  accuracy  and  in  the  subjective 
ratings.  The  median  bomb  score  decreased  from  about  160  feet  to  143  feet,  and  the  average 
composite  score  increased  from  5.4  to  6.8.  This  improvement  results  because  the  pilot’s  improved 
performance  on  the  initial  stages  (prior  to  apex)  places  him  in  a better  position  to  start  his  dive. 
Thus,  errors  in  later  stages  of  the  delivery  have  a reduced  effect  on  overall  performance,  even 
though  performance  on  these  stages  has  improved  only  slightly.  The  difference  in  the  criticality  of 
the  stages  to  success  of  the  pass  is  a result  of  the  cumulative  effect  of  error  in  the  sequence.  An 
error  made  early  in  the  pass,  perhaps  more  than  a mile  from  the  release  point,  can  result  in  a 
large  displacement  from  the  desired  position  by  the  time  the  release  point  is  reached.  An  error  of 
the  same  magnitude  made  late  in  the  delivery,  on  the  other  hand,  is  less  critical  because  the 
weapon  is  released  before  the  error  condition  can  attain  as  serious  a magnitude.  Thus,  early  errors 
have  a greater  effect  on  overall  performance  and  can  require  more  extreme  corrective  action  than 
errors  made  in  later  stages. 

Tactical  Range  Performance.  The  performance  on  the  tactical  range  (see  Table  3)  was  similar 
to  that  on  Sortie  GAT  3 (controlled  range).  The  initial  performance  on  all  task  stages  was  slightly 
lower  compared  to  performance  on  later  control  range  passes,  but  the  relative  difficulty  of  the 
stages  was  similar.  The  exception  was  target  acquisition  which  was  markedly  lower  compared  even 
to  early  control  range  performance.  This  depression  reflects  the  greater  difficulty  of  acquiring 
poorly  defined  targets  on  the  tactical  range.  By  the  end  of  the  GAT  phase,  the  pilots  attained 
considerable  skill  at  all  stages  of  the  delivery,  althdugh  performance  has  not  reached  a 100% 
satisfactory  level. 


Table  3.  Stage-by-Stage  Analysis  of  the  Pop-Up  — Tactical  Range  Performance 


Task  Breakdown:  Percent  Satisfactory  Bomb  Scores 

1 3 4 5 7 8 10  MED  MEAN 

TOT  APPTO  2 CLIMB  TGT  PULL  6 TRK  BMB  9 RTN  TO  BOMB  COMP 


GAT  PASS 

PLT 

PUP 

PUP 

LEG 

ACO 

DOWN 

APEX 

PNT 

RUN 

REC 

LO  LVL 

SCORE 

N 

RATING 

4 1 

21 

76 

81 

86 

67 

86 

86 

71 

57 

76 

57 

32 

12 

6.14 

4 2 

17 

88 

88 

88 

59 

76 

76 

71 

47 

88 

59 

50 

11 

6.13 

5 1 

21 

95 

62 

86 

90 

81 

67 

71 

62 

90 

71 

100 

14 

7.06 

5 2 

14 

86 

71 

93 

93 

79 

86 

64 

36 

79 

64 

125 

8 

5.75 

6 1 

21 

90 

81 

95 

95 

76 

81 

86 

71 

90 

90 

32 

14 

7.81 
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IV.  CONCLUSIONS 


The  pop-up  rating  methodology  was  successfully  used  to  evaluate  pilot  performance  on  the 
pop-up.  The  results  of  the  present  study  demonstrated  the  validity  of  the  performance  rating 

methodology.  Several  important  conclusions  can  be  drawn  from  the  present  results: 

1.  Subjective  ratings  of  pilot  performance  can  provide  a valid  and  reliable  measure  of  pilot 
proficiency.  This  was  shown  by  the  high  contingency  coefficient  obtained  between  the  composite 
rating  score  and  bomb  score.  This  composite  rating  score  was  obtained  from  3-point  (Satisfactory- 
Marginal-Unsatisfactory)  ratings  of  nine/ten  critical  stages  of  the  maneuver. 

2.  Ratings  of  the  individual  stages  of  the  delivery  provide  information  that  is  not  available 
from  overall  objective  performance  measures,  such  as  bomb  score.  Stage  ratings  provide  information 
about  skill  areas  which  give  pilots  initial  difficulty  and  about  the  effect  of  training  and  practice 
on  proficiency  at  skills  required  for  combat  readiness. 

3.  The  stage  analysis  permits  inferences  to  be  made  about  the  effectiveness  of  training 

programs.  In  the  present  study,  performance  on  individual  stages  of  the  pop-up  reflected  the 

amount  of  emphasis  given  to  certain  flying  skill  areas  up  to  this  point  in  the  F-4  training 

program.  These  areas  are  low  altitude  awareness/aircraft  handling  (stages  1 and  2),  instrument 
crosscheck  (stage  3),  basic  fighter  maneuvers  (stages  5 and  6),  and  basic  bombing  (stages  7 and  8). 
In  some  skill  areas,  pilots  in  the  present  study  showed  considerable  mastery.  In  other  skill  areas, 
pilots’  level  of  proficiency  was  lower. 

Low  Altitude  A wareness/ Air  craft  Handling 

During  the  approach  to  PUP,  and  PUP,  the  pilot  must  navigate  and  maneuver  the  aircraft 
under  severe  time  pressure.  This  time  stress  is  compounded  by  the  stress  of  flying  in  the  very 
unforgiving  low-altitude  environment.  Prior  to  this  phase  of  training,  pilots  have  not  performed 
high-g,  low  altitude  maneuvering  or  target  area  navigation.  As  a result,  performance  on  the  initial 
stages  of  the  pop-up  is  poor,  and  these  stages  are  not  fully  mastered  by  the  end  of  the  GAT 
phase. 

Instrument  Cross-Check 

The  climb  leg  is  basically  an  instrument  task,  involving  setting  a climb  angle  and  heading. 
This  skill  has  been  practiced  frequently  since  the  start  of  F4  transition  training.  Pilots  have  little 
difficulty  in  mastering  it  in  the  GAT  phase. 

Basic  Fighter  Maneuvers  (BFM) 

Skill  at  BFM,  that  is,  the  capacity  to  maneuver  the  aircraft  by  feel,  is  exercised  primarily  in 
the  medium-  to  high-altitude  combat  environment.  As  Ihe  F4  has  been  phased  out  of  the 
air-to-air  role,  emphasis  on  BFM  in  the  syllabus  has  been  reduced.  The  apex  and  pull-down  stages 
both  draw  heavily  on  BFM  skills.  These  stages  are  performed  poorly  and  are  not  completely 

mastered  during  the  GAT  phase. 

Basic  Bombing 

The  basic  bombing  skill  is  initially  acquired  during  the  GA  phase,  which  precedes  the  GAT 
phase.  At  the  start  of  the  GAT  phase,  it  is  the  skill  area  which  gives  pilots  the  greatest  difficulty, 
and  at  the  end  of  the  phase,  it  remains  the  most  difficult.  In  fact,  the  track-point  and  bomb-run 
stages  show  the  least  improvement  of  any  during  the  GAT  phase.  This  is  not  too  surprising,  since 
the  pilot  has  his  hands  full  with  many  new  and  difficult  tasks  prior  to  reaching  the  basic 

bombing  stages  of  the  pop-up.  The  interviews  with  pilots  and  the  rating  data  suggest  that  basic 

bombing  skills  are  more  effectively  acquired  when  they  are  practiced  in  relative  isolation  (e.g., 
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during  the  GA  phase).  Thus,  improving  performance  on  the  track  point  and  bomb  run  stages 
could  best  be  accomplished  by  increasing  the  basic  bombing  training  given  in  the  GA  phase. 

B-Course  Syllabus 

The  Tactical  Air  Command  has  recently  modified  the  B-course  training  syllabus  in  an  effort 
unrelated  to  the  present  research.  These  changes  have  involved  adding  low  altitude  awareness 
training  (LAAT)  prior  to  the  GAT  phase,  increasing  the  amount  of  basic  bombing  in  the  GA 
phase,  and  decreasing  the  amount  of  BFM.  These  changes  can  be  expected  to  impact  plot 
performance  in  learning  the  pop-up  during  the  GAT  phase.  The  addition  of  LAAT  may  result  in 
greater  pilot  proficiency  on  approach  to  PUP  and  pull-up.  Increasing  basic  bombing  may  lead  to 
improved  performance  on  the  track  point  and  bomb  run  stages  of  the  pop-up,  and  the  decreased 
emphasis  on  BFM  might  be  expected  to  lead  to  poorer  performance  on  pull-down  and  apex.  A 
study  is  planned  to  address  these  questions  when  pilots  in  the  modified  B-course  reach  GAT 
phase. 
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APPENDIX  A : 


DATA  COLLECTION  MATERIALS 


PILOT  #: 
EVENT: 


F-4  GAT  PHASE  EVALUATION 
IP  #:  A RANGE  #: 


DATE: 


RIDE  #: 


Release  Parameters 

Estimated  Dive  Angle 
Estimated  Altitude  (AGL) 
Estimated  Airspeed 
Bomb  Score 

TASK  EVALUATION 
1.  Approach  to  PUP 


PASS  # 


1st 


2nd 


jrd 


4th 


COMMENTS/INDICATE  PASS  # 


2.  PUP 


3.  Climb  Leg 


4.  Target  Acquisition 


5.  Pull  Down  Point 


6.  Apex 


7.  Track  Point 


8.  Bomb  Run 


9.  Recovery 


10.  Return  to  Low  Level 


The  pop-up  evaluation  form  was  developed  to  assess  student  pilot  performance 
on  pop-up  weapon  deliveries.  Additionally,  this  form  can  be  used  as  a 
briefing  and  debriefing  aid.  Tape  recorders  will  be  provided  to  assist  the 
crew  in  gathering  data  pertinent  to  completion  of  the  form. 

INSTRUCTIONS 

1.  COMPLETE  THE  PILOT  IDENTIFICATION  PORTION  OF  THE  FORM 

2.  RECORD  ESTIMATED  RELEASE  PARAMETERS  AND  BOMB  SCORES. 

3.  GRADE  THE  TASK  EVALUATION  SECTION  AS  FOLLOWS: 

S - Satisfactory.  Task  performance  met  criteria  with  minimal  error; 
minimal  compensations  were  required. 

M - Marginal.  Task  performance  met  criteria  with  error;  compen- 
sations were  required  to  salvage  the  pass  delivery. 

U - Unsatisfactory.  Task  performance  did  not  meet  criteria;  gross 
errors  in  performance  led  to  either  an  unsafe  or  aborted  pass. 

Any  item  graded  as  either  M or  U requires  an  appropriate  explanation  under 
the  comments  section.  The  following  indicates  those  requirements  identified 
with  each  item  included  in  the  Task  Evaluation  section: 

1.  Approach  to  PUP:  (a)  Acquisition  of  PUP;  (b)  Altitude  control;  and 
(c)  Airspeed  control. 

2.  PUP:  (a)  Heading  correction;  (b)  "G"  application;  (c)  Airspeed 
correction. 

3.  Climb  Leg:  (a)  Climb  angle  corrections. 

4.  Target  Acquisition:  Self-explanatory 

5.  Pull  Down  Point:  (a)  Roll;  (b)  Airspeed  corrections;  and  (c)  "G" 
application. 

6.  Apex:  (a)  Pattern  correction;  and  (b)  Airspeed  corrections. 

7.  Track  Point:  (a)  Aim  off  point;  (b)  Roll  out;  (c)  Initial  wind 
correction;  and  (d)  Angle  check. 

8.  Bomb  Run:  (a)  Aiming  error  corrections;  (b)  Airspeed  control;  and  (c) 
Exposure  time  control. 

9.  Recovery:  (a)  "G"  application;  and  (b)  Jinking. 

10.  Return  to  Low  Level:  (a)  Exposure  time;  and  (b)  Transition  to  low  level. 
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